Working with Beliefs: AI Transparency in the Enterprise

Enterprises are increasingly recognizing that they must integrate AI into all of their operational workflows to remain competitive. As enterprises consider competing AIs to support a particular business function, explainability is an advantage which gets a candidate AI a foot in the door. Our experience working with enterprise decision makers considering AI in a decision augmentation role reveals an additional and possibly more crucial aspect of choosing an AI: the ability of decision makers to interact fluidly with an AI. Fluid interactions are necessary when an AI’s recommendation does not match a human decision maker’s existing beliefs. Interactions that allow the (typically nontechnical) human to edit the AI, as well as allow the AI to guide the human, enable a collaborative exploration of the data that leads to common ground where both the AI and the human beliefs have been updated. We outline an illustrative example from our experience that models this dance. Based on our experiences, we suggest requirements for AI systems that would greatly facilitate their adoption in the enterprise. Author